Results of the IROCA international clinical audit in prostate cancer radiotherapy at six comprehensive cancer centres.

To assess adherence to standard clinical practice for the diagnosis and treatment of patients undergoing prostate cancer (PCa) radiotherapy in four European countries using clinical audits as part of the international IROCA project. Multi-institutional, retrospective cohort study of 240 randomly-selected patients treated for PCa (n = 40/centre) in the year 2015 at six European hospitals. Clinical indicators applicable to general and PCa-specific radiotherapy processes were evaluated. All data were obtained directly from medical records. The audits were performed in the year 2017. Adherence to clinical protocols and practices was satisfactory, but with substantial inter-centre variability in numerous variables, as follows: staging MRI (range 27.5-87.5% of cases); presentation to multidisciplinary tumour board (2.5-100%); time elapsed between initial visit to the radiation oncology department and treatment initiation (42-102.5 days); number of treatment interruptions ≥ 1 day (7.5-97.5%). The most common deviation from standard clinical practice was inconsistent data registration, mainly failure to report data related to diagnosis, treatment, and/or adverse events. This clinical audit detected substantial inter-centre variability in adherence to standard clinical practice, most notably inconsistent record keeping. These findings confirm the value of performing clinical audits to detect deviations from standard clinical practices and procedures.

A quality improvement programme in radiotherapy using workflow audits.

BACKGROUND: Evaluation of the quality of healthcare depends on measures of structures, processes and outcomes. Progress in recording data allows for better measures of processes, such as the completeness of clinical data, the performance of professional tasks and the use of checklists. OBJECTIVES: To report the results of a radiotherapy (RT) workflow audit and a subsequent online survey of user experience. METHODS: The RT workflow audit was developed in 2016 and has been undertaken twice a year at 28 facilities or units, with a total of 32 linear accelerators. Electronic patient folders were reviewed to assess the documentation of 90 task items, of which 64 were scored. The auditor came from another facility. The online survey took place in July 2020. It contained questions on the audit's process, professional value and future use. Invitations were sent by email to the 151 radiotherapist staff at the 28 units where the audit had been implemented. Responses were anonymous. RESULTS: For the RT workflow audit, scores improved from 60% in some units in 2016 to >90% in all units for at least 2 years since 2018. The number of responders to the online survey was 58, giving a responder rate of 38%. The margin of error of the results was 10%. The audit's task items were considered appropriate by 77% of responders, and feedback was reported by 78% of them. The audit was considered very or extremely valuable to their unit's service delivery by 58% of responders. Changes in the unit as a result of the audit were reported by 77% of responders. The audit was very useful or extremely useful to 75% of responders in maintaining personal professional standards. The proportion of responders who were very or extremely supportive of continuing with the audit was 77%. The comments in the online survey will be helpful for ongoing review of the RT workflow audit. CONCLUSIONS: The RT workflow audit extends the scope of accreditation audits by including measures of processes. Users of the audit evaluate its processes favourably and report that it has value both in their unit's clinical service and for their personal professional standards. The audit is effective in developing quality improvement programmes.

Improving quality of preclinical academic research through auditing: A feasibility study.

How much can we rely on whether what was reported in a study was actually done? Systematic and independent examination of records, documents and processes through audits are a central element of quality management systems. In the context of current concerns about the robustness and reproducibility of experimental biomedical research audits have been suggested as a remedy a number of times. However, audits are resource intense and time consuming, and due to their very nature may be perceived as inquisition. Consequently, there is very little experience or literature on auditing and assessments in the complex preclinical biomedical research environment. To gain some insight into which audit approaches might best suit biomedical research in academia, in this study we have applied a number of them in a typical academic neuroscience environment consisting of twelve research groups with about 100 researchers, students and technicians, utilizing the full gamut of state-of-the-art methodology. Several types of assessments and internal as well as external audits (including the novel format of a peer audit) were systematically explored by a team of quality management specialists. An experimental design template was developed (and is provided here) that takes into account and mitigates difficulties, risks and systematic errors that may occur during the course of a study. All audits were performed according to a pre-defined workflow developed by us. Outcomes were assessed qualitatively. We asked for feedback from participating employees in every final discussion of an audit and documented this in the audit reports. Based on these reports follow-up audits were improved. We conclude that several realistic options for auditing exist which have the potential to improve preclinical biomedical research in academia, and have listed specific recommendations regarding their benefits and provided practical resources for their implementation (e.g. study design and audit templates, audit workflow).

[Establishment of an On-Site Dosimetric Audit Program with a Plastic Phantom].

The quality assurance (QA) of the output constancy of linear accelerators (linacs) is one of the most basic items for radiation therapy. The necessity and demand of the dosimetric audit is on the rise. We aimed at establishing an on-site dosimetric audit program with a plastic phantom in order to enrich the availability of dosimetric audit in Japan. By this, we developed and evaluated an on-site audit program in multiple institutions in 120 cases for photon and electron standard dosimetry and intensity modulated radiation therapy (IMRT) condition. For photon standard dosimetry, we evaluated the accuracies of institutional measured absorbed dose in 50 cases by this on-site audit program. For the electron standard dosimetry, we evaluated the accuracies of institutional measured absorbed dose in 25 cases. For IMRT condition, we evaluated the accuracies of the institutional calculated dose in radiation treatment planning systems in 45 cases. The agreements of the measured absorbed dose between our audit and the institutions were within±1.1% for photon standard dosimetry. The agreements of the measured absorbed dose between our audit and the institutions were within±2.1% for electron standard dosimetry. The agreements between the measured absorbed dose of our audit and the institutional calculated dose were within±2.1% for IMRT condition. We established an on-site dosimetric audit program with a plastic phantom and this program is suitable with tight criteria similar to criteria required for clinical QA.

Impetus to change: a multi-site qualitative exploration of the national audit of dementia.

BACKGROUND: National audit is a key strategy used to improve care for patients with dementia. Audit and feedback has been shown to be effective, but with variation in how much it improves care. Both evidence and theory identify active ingredients associated with effectiveness of audit and feedback. It is unclear to what extent national audit is consistent with evidence- and theory-based audit and feedback best practice. METHODS: We explored how the national audit of dementia is undertaken in order to identify opportunities to enhance its impact upon the improvement of care for people with dementia. We undertook a multi-method qualitative exploration of the national audit of dementia at six hospitals within four diverse English National Health Service organisations. Inductive framework analysis of 32 semi-structured interviews, documentary analysis (n = 39) and 44 h of observations (n = 36) was undertaken. Findings were presented iteratively to a stakeholder group until a stable description of the audit and feedback process was produced. RESULTS: Each organisation invested considerable resources in the audit. The audit results were dependent upon the interpretation by case note reviewers who extracted the data. The national report was read by a small number of people in each organisation, who translated it into an internal report and action plan. The internal report was presented at specialty- and organisation-level committees. The internal report did not include information that was important to how committee members collectively decided whether and how to improve performance. Participants reported that the national audit findings may not reach clinicians who were not part of the specialty or organisation-level committees. CONCLUSIONS: There is considerable organisational commitment to the national audit of dementia. We describe potential evidence- and theory-informed enhancements to the enactment of the audit to improve the local response to performance feedback in the national audit. The enhancements relate to the content and delivery of the feedback from the national audit provider, support for the clinicians leading the organisational response to the feedback, and the feedback provided within the organisation.

Long-Term Assessment of the Cecal Intubation Rates in High-Performing Colonoscopists: Time for Review.

OBJECTIVES: The cecal intubation rate (CIR) is one of the 3 priority indicators for quality in colonoscopy. Whether continuous measurement of CIR is useful in high performers is uncertain. METHODS: At an academic center, we identified 16 physicians who performed at least 50 procedures over 6 consecutive years. We analyzed all colonoscopy procedures excluding those with poor/inadequate preparation or severe colitis for CIR trend over the years. We calculated the numbers needed to establish CIR over minimum threshold levels with 95% confidence. RESULTS: The overall CIR was 99.4%. None of the 16 physicians had a CIR <96.6% in any year. Sensitivity analyses including patients without intent to reach the cecum and inadequate bowel preparation had little impact on the results. Overall cecal photo documentation rate was 98.4%. No significant correlation was observed between procedure volume at our center and CIR (σ = -0.196, P = 0.483). Physicians with CIR ≥99% need to have only 24 examinations reviewed to establish CIR is >95%. DISCUSSION: Continuous measurement of CIR, at least in high performers, appears to be of limited value. Very high performers need to evaluate small number of cases to demonstrate that CIR is above the recommended thresholds.

A clinical audit of oral anticoagulant therapy in aged care residents with atrial fibrillation.

Background Atrial fibrillation (AF) is a risk factor for stroke in older people. Oral anticoagulants can reduce stroke risk but they are commonly under prescribed in the elderly, often due to concerns regarding the risk of bleeding. Prescribing in aged care residents may also be further complicated by associated geriatric conditions such as dementia and risk of falls. Systematic assessment of stroke and bleed risk can help identify people with AF expected to benefit from anticoagulant therapy and optimise prescribing. Objective The aim of this study was to assess the prescribing of anticoagulants in elderly aged care residents in accordance with Australian guideline recommendations. Setting Nineteen aged care facilities across southeast Queensland. Method A clinical audit of anticoagulant therapy in aged care residents was conducted. Main outcome measure Information was collected from the records of residents with non-valvular AF to assess the risk of stroke and bleeding and compare this prescribing to current evidence-based guidelines for anticoagulation in AF. Results A total of 1754 residents were screened with 359 (20.4%) identified to have a diagnosis of non-valvular AF. There were 356 (99.2%) residents with non-valvular AF and a sufficiently high risk of stroke to warrant the use of an anticoagulant. Of these, 172 (48.3%) were prescribed an oral anticoagulant and 40 (11.2%) residents had a documented decision not to prescribe oral anticoagulants in their records. The majority of residents prescribed anticoagulation were receiving non-vitamin K antagonists (76.8%). The prescribed dose was consistent with recommendations for 44.8% of residents prescribed oral anticoagulant therapy. Conclusion Many residents with non-valvular AF and significant risk of stroke were not receiving oral anticoagulation despite a lack of documented reason for non-prescribing. Non-vitamin K antagonists were widely prescribed but dosing of these agents could still be improved. There remains a need to improve anticoagulant prescribing for aged care residents and optimise stroke prevention in this population.

50 Years of the IAEA/WHO postal dose audit programme for radiotherapy: what can we learn from 13756 results?

Background: The IAEA/WHO postal dose audit programme has been operating since 1969 with the aim of improving the accuracy and consistency of dosimetry in radiotherapy in low-income and middle-income countries world-wide. This study summarises the 50 years' experience of audits and explores the quality of reference dosimetry in participating radiotherapy centres throughout the years.Material and methods: During the IAEA/WHO postal audits the dose determined from the mailed dosimeter is compared with that stated by the participant. Agreement to within ±5% is regarded acceptable whilst deviations outside ±5% limits trigger follow-up actions. Of particular interest in this study was the dependence of clinical dosimetry quality on factors related to the centre infrastructure and expertise in dosimetry of its staff.Results: The IAEA/WHO dose audit programme noted great increase in the overall percentage of acceptable results from about 50% in its early years to 99% at present, although there is some variability of results amongst participating countries. Whereas results for younger radiotherapy machines show the agreement rate between the measured and the stated doses well above 90%, for those over 20 years old the rate dropped to <80%. Linac dosimetry was always better than 60Co dosimetry and multi-machine centres generally performed better than single machine centres equipped with cobalt alone. Second and subsequent participation in audits showed higher quality dosimetry than the first participation. The implementation of modern dosimetry protocols resulted in more accurate dosimetry than the use of the older protocols.Conclusions: Over the 50 years that the IAEA has accumulated dosimetry audit data, practices in radiotherapy centres have significantly improved. Higher quality dosimetry confirmed in audits is generally associated with better infrastructure and adequate dosimetry expertise of medical physicists in participating centres.

An IAEA survey of radiotherapy practice including quality assurance extent and depth.

Background: The IAEA recommends a quality assurance program in radiotherapy to ensure safe and effective treatments. In this study, radiotherapy departments were surveyed on their current practice including the extent and depth of quality assurance activities.Methods: Radiotherapy departments were voluntarily surveyed in three stages, firstly, in basic facility information, secondly, in quality assurance activities and treatment techniques, and thirdly, in a snapshot of quality assurance, departmental and treatment activities.Results: The IAEA received completed surveys from 381 radiotherapy departments throughout the world with 100 radiotherapy departments completing all three surveys. Dominant patterns were found in linac-based radiotherapy with access to treatment planning systems for 3D-CRT and 3D imaging. Staffing levels for major staff groups were on average in the range recommended by the IAEA. The modal patient workload per EBRT unit was as expected in the range of 21-30 patients per day, however significant instances of high workload (more than 50 patients per day per treatment unit) were reported. Staffing levels were found to correlate with amount of treatment equipment and patient workload. In a self-assessment of quality assurance performance, most radiotherapy departments reported that they would perform at least 60% of the quality assurance activities itemized in the second survey, with particular strength in equipment quality control. In a snapshot survey of quality assurance performance, again equipment quality control practice was well developed, particularly for the treatment equipment.Conclusions: The IAEA surveys provide a snapshot of current radiotherapy practice including quality assurance activities.

High-Risk Emergency Laparotomy in Australia: Comparing NELA, P-POSSUM, and ACS-NSQIP Calculators.

BACKGROUND: The National Emergency Laparotomy Audit (NELA) highlights the importance of identifying high-risk patients due to the potential for significant morbidity and mortality. The NELA risk prediction calculator (NRPC) was developed from data in England and Wales and is one of several calculators available. We seek to determine the utility of NRPC in the Australian population and compare it with Portsmouth Physiological and Operative Severity Score for the enumeration of mortality and Morbidity (P-POSSUM) and American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) calculators. METHODS: A retrospective review of all emergency laparotomies undertaken at four Australian centers was performed between January 2016 and December 2017. Data extracted from patient records were used to calculate NRPC, ACS-NSQIP, and P-POSSUM scores for 30-day mortality risk. The sensitivity of NRPC was assessed using the NELA high-risk cohort score of ≥10% and this was compared with the other two calculators. RESULTS: There were 562 (M = 261, mean age = 66 [±17] y) patient charts reviewed in the study period. 59 patients died within 30 d (10.5%). NRPC was able to identify 52 (sensitivity = 88.1%) of these as being within the high-risk group. Using the NELA high-risk cutoff, NRPC identified 52 deaths of 205 (25.4%) high-risk patients, P-POSSUM identified 46 of 245 (18.8%), and ACS-NSQIP identified 46 of 201 (22.9%). Using the McNemar test, no significant difference was noted between NRPC and P-POSSUM (P = 0.07) or NRPC and ACS-NSQIP (P = 0.18). CONCLUSIONS: In the Australian context, the NRPC is a highly sensitive and useful tool for predicting 30-day mortality in high-risk emergency laparotomy patients and is comparable with P-POSSUM and ACS-NSQIP calculators.

The use of real-world data to address questions of patient safety.

Pharmacovigilance registries of biologics were established to evaluate the risk of adverse events that may be missed in trials due to shorter durations and homogeneous samples. This review will present the strengths and weaknesses of registry data in addressing patient safety issues. Since their inception, scope has broadened because registries represent a relatively inexpensive approach to answering many clinical questions, both research and non-research focused. They achieve high statistical power, allow direct comparability, and offer a level of detail about adverse events not possible with trial data. Registries have been central in clarifying the risk of infection and malignancy with anti-TNF therapy, despite the limitations of selection and channelling bias, incomplete case capture, unmeasured confounding, and the inability to infer causality. Routinely collected data from electronic health records and national audits offer alternative real-world resources, further assisting patients and clinicians in understanding the risks of biologic therapy choices.

IAEA methodology for on-site end-to-end IMRT/VMAT audits: an international pilot study.

Background: The IAEA has developed and tested an on-site, end-to-end IMRT/VMAT dosimetry audit methodology for head and neck cases using an anthropomorphic phantom. The audit methodology is described, and the results of the international pilot testing are presented.Material and methods: The audit utilizes a specially designed, commercially available anthropomorphic phantom capable of accommodating a small volume ion chamber (IC) in four locations (three in planning target volumes (PTVs) and one in an organ at risk (OAR)) and a Gafchromic film in a coronal plane for the absorbed dose to water and two-dimensional dose distribution measurements, respectively. The audit consists of a pre-visit and on-site phases. The pre-visit phase is carried out remotely and includes a treatment planning task and a set of computational exercises. The on-site phase aims at comparing the treatment planning system (TPS) calculations with measurements in the anthropomorphic phantom following an end-to-end approach. Two main aspects were tested in the pilot study: feasibility of the planning constraints and the accuracy of IC and film results in comparison with TPS calculations. Treatment plan quality was scored from 0 to 100.Results: Forty-two treatment plans were submitted by 14 institutions from 10 countries, with 79% of them having a plan quality score over 90. Seventeen sets of IC measurement results were collected, and the average measured to calculated dose ratio was 0.988 ± 0.016 for PTVs and 1.020 ± 0.029 for OAR. For 13 film measurement results, the average gamma passing rate was 94.1% using criteria of 3%/3 mm, 20% threshold and global gamma.Conclusions: The audit methodology was proved to be feasible and ready to be adopted by national dosimetry audit networks for local implementation.

A Collaborative Multicenter QI Initiative to Improve Antibiotic Stewardship in Newborns.

OBJECTIVES: To determine if NICU teams participating in a multicenter quality improvement (QI) collaborative achieve increased compliance with the Centers for Disease Control and Prevention (CDC) core elements for antibiotic stewardship and demonstrate reductions in antibiotic use (AU) among newborns. METHODS: From January 2016 to December 2017, multidisciplinary teams from 146 NICUs participated in Choosing Antibiotics Wisely, an Internet-based national QI collaborative conducted by the Vermont Oxford Network consisting of interactive Web sessions, a series of 4 point-prevalence audits, and expert coaching designed to help teams test and implement the CDC core elements of antibiotic stewardship. The audits assessed unit-level adherence to the CDC core elements and collected patient-level data about AU. The AU rate was defined as the percentage of infants in the NICU receiving 1 or more antibiotics on the day of the audit. RESULTS: The percentage of NICUs implementing the CDC core elements increased in each of the 7 domains (leadership: 15.4%-68.8%; accountability: 54.5%-95%; drug expertise: 61.5%-85.1%; actions: 21.7%-72.3%; tracking: 14.7%-78%; reporting: 6.3%-17.7%; education: 32.9%-87.2%; P < .005 for all measures). The median AU rate decreased from 16.7% to 12.1% (P for trend < .0013), a 34% relative risk reduction. CONCLUSIONS: NICU teams participating in this QI collaborative increased adherence to the CDC core elements of antibiotic stewardship and achieved significant reductions in AU.

First evidence of the clinical impact of a general practitioner with a special interest in epilepsy.

PURPOSE: The purpose of the study was to review the clinical outcomes of people with epilepsy (PWE) attending a primary care-based specialist epilepsy service. METHOD: The case notes of 355 people attending the service and subsequent follow-up from 2005 to 2013 were reviewed. RESULTS: There had been 37 deaths (all nonattributable to epilepsy), and 38 people had left the area, leaving 280 people who completed the audit. Positive outcomes could be attributed in 94% still attending the service at the end of follow-up. Seventy-five percent of people on treatment, referred with poor seizure control, achieved seizure remission with antiepilepsy drug (AED) changes initiated by the service. CONCLUSION: This study suggests that the majority of people who attended the service had a positive outcome and provides the first evidence for the clinical effectiveness of a general practitioner (GP) with special interest in epilepsy (GPwSIe) and provides support for the recommendations in earlier government reports to promote the use of such a service. Clinical Commissioning Groups (CCGs) and Government should consider investment in this intermediate tier of care as a means to both improving the quality of care and potentially reducing costs.

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care.

BACKGROUND: Audit and feedback (A&F) is more effective if it facilitates action planning, but little is known about how best to do this. We developed an electronic A&F intervention with an action implementation toolbox to improve pain management in intensive care units (ICUs); the toolbox contained suggested actions for improvement. A head-to-head randomised trial demonstrated that the toolbox moderately increased the intervention's effectiveness when compared with A&F only. OBJECTIVE: To understand the mechanisms through which A&F with action implementation toolbox facilitates action planning by ICUs to increase A&F effectiveness. METHODS: We extracted all individual actions from action plans developed by ICUs that received A&F with (n = 10) and without (n = 11) toolbox for 6 months and classified them using Clinical Performance Feedback Intervention Theory. We held semi-structured interviews with participants during the trial. We compared the number and type of planned and completed actions between study groups and explored barriers and facilitators to effective action planning. RESULTS: ICUs with toolbox planned more actions directly aimed at improving practice (p = 0.037) and targeted a wider range of practice determinants compared to ICUs without toolbox. ICUs with toolbox also completed more actions during the study period, but not significantly (p = 0.142). ICUs without toolbox reported more difficulties in identifying what actions they could take. Regardless of the toolbox, all ICUs still experienced barriers relating to the feedback (low controllability, accuracy) and organisational context (competing priorities, resources, cost). CONCLUSIONS: The toolbox helped health professionals to broaden their mindset about actions they could take to change clinical practice. Without the toolbox, professionals tended to focus more on feedback verification and exploring solutions without developing intentions for actual change. All feedback recipients experienced organisational barriers that inhibited eventual completion of actions. TRIAL REGISTRATION: ClinicalTrials.gov, NCT02922101 . Registered on 26 September 2016.

Testing the methodology for a dosimetric end-to-end audit of IMRT/VMAT: results of IAEA multicentre and national studies.

Introduction: Within an International Atomic Energy Agency (IAEA) co-ordinated research project (CRP), a remote end-to-end dosimetric quality audit for intensity modulated radiation therapy (IMRT)/ volumetric arc therapy (VMAT) was developed to verify the radiotherapy chain including imaging, treatment planning and dose delivery. The methodology as well as the results obtained in a multicentre pilot study and national trial runs conducted in close cooperation with dosimetry audit networks (DANs) of IAEA Member States are presented.Material and methods: A solid polystyrene phantom containing a dosimetry insert with an irregular solid water planning target volume (PTV) and organ at risk (OAR) was designed for this audit. The insert can be preloaded with radiochromic film and four thermoluminescent dosimeters (TLDs). For the audit, radiotherapy centres were asked to scan the phantom, contour the structures, create an IMRT/VMAT treatment plan and irradiate the phantom. The dose prescription was to deliver 4 Gy to the PTV in two fractions and to limit the OAR dose to a maximum of 2.8 Gy. The TLD measured doses and film measured dose distributions were compared with the TPS calculations.Results: Sixteen hospitals from 13 countries and 64 hospitals from 6 countries participated in the multicenter pilot study and in the national runs, respectively. The TLD results for the PTV were all within ±5% acceptance limit for the multicentre pilot study, whereas for national runs, 17 participants failed to meet this criterion. All measured doses in the OAR were below the treatment planning constraint. The film analysis identified seven plans in national runs below the 90% passing rate gamma criteria.Conclusion: The results proved that the methodology of the IMRT/VMAT dosimetric end-to-end audit was feasible for its intended purpose, i.e., the phantom design and materials were suitable; the phantom was easy to use and it was robust enough for shipment. Most importantly the audit methodology was capable of identifying suboptimal IMRT/VMAT delivery.

To MSNAP or not to MSNAP? Testing a small regional memory clinic against the UK Memory Service National Accreditation Program (MSNAP).

AimTo investigate whether a small regional memory clinic would benefit from engaging with a structured external audit process such as the Royal College of Psychiatrists' Memory Service National Accreditation Program (MSNAP). BACKGROUND: The Psychiatry of Old Age service in Navan operates a public cognitive clinic. Despite the publication of the 2014 National Dementia Strategy, there are currently no national standards for memory clinics in Ireland. It may be beneficial to link in with an external quality control system as part of routine clinical governance. METHODS: Published data from the MSNAP group was reviewed and a set of audit materials extrapolated to replicate the MSNAP self-review process. The audit cycle involved (1) retrospective case review, (2) institution of a range of interventions and (3) a prospective audit, which included service user feedback. RESULTS: Overall the results demonstrated a high standard of service, especially in the areas of accessibility, assessment and communication of diagnosis. The clinic performed well against MSNAP key performance indicators. Patient and carer satisfaction with the service was very high. Clinic policies needed further development, particularly in the areas of referral, consent and data protection. CONCLUSIONS: The process was useful, providing clear pointers for action. It highlighted the need to formalise organisational and practice policies, patient support and education, audit and outreach. Although accreditation is a laborious process requiring financial investment, it provides a strong scaffold to maintain and improve standards and is likely to be a valuable learning experience, where national guidelines are lacking.